Function Approximation using Dividing Features

Automated agents designed to learn strategies using Markov decision processes on a continuous state space usually need to approximate the value function associated with the environment. Traditionally, as described in Sutton and Barto in [1], this is often done using a fixed number of rectangular features tiled across the state space, possibly distributed into multiple layers. As an improvement to this concept, we propose an algorithm that allows the number of features to grow as the agent learns more about the environment, thus enabling the agent to adapt to the particular circumstances and to improve the effectiveness of each feature. Experimental results obtained by running the traditional agent and our improved version show that our agent can indeed learn a better strategy using fewer time steps.